CACI Helps U.S. Navy Reserve Increase Readiness with Mobile Q&A Chatbot Built on AWS

Since 1915, the United States Navy Reserve has provided operational and military support to the United States Navy, Marine Corps, and Joint forces in peacetime and in war. The Navy Reserve consists of more than 56,000 officers and enlisted personnel who serve in every state and territory as well as across the globe. The main priority of the Navy Reserve is warfighting readiness for its Reservists. “We strive to support readiness by making things more efficient for Reserve Sailors, because they likely have full-time jobs and we need to do our best to fully engage them in their Reserve life,” says Joel Stewart, Commander, Navy Reserve Forces Command (CNRFC) N65 division director, United States Navy Reserve. CNRFC N65 is the IT development division of the Navy Reserve.

Ensuring readiness has become increasingly challenging as the volume of critical information has grown. “Reservists have to navigate and understand an overwhelming number of complex processes, policies, and procedures, and they don’t always have time to read through all these instructional and guidance documents,” Stewart says. The Navy Reserve sought to use machine learning to solve the problem. Specifically, the organization wanted to leverage large language models (LLMs) to comprehend and generate text from policy documents. Stewart says, “By using LLMs, we thought we could make documents more legible to increase Reservist engagement and ultimately increase the state of readiness.” 

To support this effort, the Navy Reserve sourced ideas from Reservists as part of its annual i3 Waypoints challenge. The 2023 winning idea was the Reserve Artificial Intelligence Navigator (RAIN), which proposed the creation of a publicly accessible, mobile Q&A chatbot that could answer standard, process-related questions for Reservists. Turning the solution into reality was potentially challenging because Navy Reserve security requirements prevent the use of commercially available hosted services. “We wanted to build a mobile app that used a third-party backend service, but we can’t do that as a government agency because it wouldn’t be accredited for maintaining data outside our infrastructure,” says Stewart.

CNRFC N65 decided to work with CACI, an AWS Partner, to turn RAIN into a mobile Q&A chatbot application. CACI, which provides solutions and services for federal government agencies, has developed and maintained business systems for the Navy Reserve for more than a decade. Tasked with developing a new solution, CACI leveraged its Strategic Collaboration Agreement (SCA) with Amazon Web Services (AWS) for assistance getting started. An AWS SCA is a strategic agreement that fosters a closer working relationship between AWS and a customer or partner, enabling them to leverage each other's strengths and expertise to achieve shared goals and drive innovation in the cloud computing space. “CACI is leaning heavily into generative AI technology, and we are familiar with the expertise AWS has in AI technologies through our relationship,” says R. J. Lewis, Software Engineering Lead at CACI. Supporting CACI via the SCA, AWS Senior Solutions Architect Dave Horne quickly grasped the problem and expertly guided CACI through an evaluation of relevant AWS services and solutions. One of those solutions is QnABot on AWS, a ready-to-deploy multi-channel and multi-language chatbot that integrates with Amazon Kendra, an intelligent enterprise search service.

CACI used Amazon Kendra to incorporate semantic search capabilities into a Retrieval-Augmented Generation (RAG) workflow within the new app. “QnABot on AWS gave us a lot of what we needed, and we liked the integration with Amazon Kendra,” says Jared Shields, Technical Lead at CACI. “We used some core tenets of the QnABot architecture, and then we carried forward those capabilities and knowledge into building our own chatbot solution.” CACI implemented the chatbot in AWS GovCloud (US), which is designed to enable government agencies to host sensitive workloads. “Although we do not currently process Controlled Unclassified Information (CUI) with our solution, we built within AWS GovCloud to ensure we’re ready when the time came,” Shields says.

To accelerate development, CACI developers used Amazon SageMaker deep learning containers with an embedded text generation interface and AWS Lambda to run code in a serverless compute environment. “Amazon Kendra helped us speed up development of our RAG pipeline because it simplified document ingestion and made it easier for us to perform semantic searching against unstructured documents,” says Shields. “Similarly, Amazon SageMaker made it easy for us to get an LLM up and running in a few hours, a task that would have taken us significantly longer without the real-time inference services provided by Amazon SageMaker. By relying on services like Amazon Kendra and Amazon SageMaker, we were able to create and deploy the application in about four months.”